UAIC Participation at ImageCLEF 2012 Photo Annotation Task

This paper presents the participation of our group in the ImageCLEF 2012 Photo Annotation Task. Our approach is based on visual and textual features as we experiment with different strategies in order to extract the semantics inside an image. First, we construct a textual dictionary of tags using the most frequent words present in the user tag annotated images from the training data sets. A linear kernel is then developed based on this dictionary. To gather more information from the images we further extract local and global visual features using TopSurf and Profile Entropy Features as well as Color Moments technique. We then aggregate these features with Support Vector Machines classification algorithm and train separate SVM models for each concept. In the end, to improve our system’s performance, we add a postprocessing step that verifies the consistency of the predicted concepts and also applies a face detection algorithm in order to increase the recognition accuracy of the person related concepts. Our submission consists of one visual-only and four multi-modal runs. We further give a more detailed perspective of our system and discuss our results and conclusions.